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ABSTRACT 

This  report  details  the  results  of  the  first  year  of  analyti¬ 
cal  and  experimental  investigations  of  programmable  optical 
quadratic  neural  networks.  The  investigations  have  included:  (1) 
computer  simulations  and  theoretical  characterizations  of  the 
performances  of  first  and  second  order  Hopfield  associative 
memories  in  terms  of  a  signal-to-noise  ratio  parameter  C;  (2)  a 
hybrid  electro-optical,  polarization-encoding-based  technique  for 
implementing  a  quadratic  neural  processor  and  (3)  the  use  of 
photorefractive  BaTi03  crystals  to  perform  a  vector-matrix-vector 
operation  based  on  four-wave  mixing.  The  details  are  summarized 
in  this  report  and  in  the  publications  resulting  from  the  research 
effort. 
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RESEARCH  OBJECTIVES 


During  the  first  year  of  the  grant  (December  1,  1987  -  Novem¬ 
ber  30,  1988) ,  the  major  research  objectives  have  been  to  perform 
both  analytical  and  experimental  investigations  of  optical  qua¬ 
dratic  neural  networks.  The  major  areas  of  investigation  have 
been  (1)  theoretical  analyses  and  computer  simulations  comparing 
the  performances  of  linear  versus  quadratic  neural  networks;  (2)  a 
hybrid  electro-optical  implementation,  based  on  liquid  crystal 
television  (LCTV)  technology  of  a  polarization-encoded  neural 
processor  ana  (3)  a  real-time  quadratic  neural  processor  utilizing 
the  nonlinear  properties  of  photorefractive  BaTi03  crystals  to 
perform  vector-matrix-vector  polynomial  operations.  Details  of 
these  first  year  investigations  are  presented  in  the  following 
sections. 
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SUMMARY  OF  RESULTS 


Since  most  of  the  results  obtained  under  the  grant  are 
promptly  submitted  for  publication,  and  are  also  presented  at 
national  and  international  scientific  meetings,  we  will  briefly 
summarize  the  major  results  obtained  in  this  section,  with  refer¬ 
ences  to  the  appropriate  journal  articles  and  conference  proceed¬ 
ings. 

1.  Theoretical/Computer  Analyses  and  Simulations  of  Linear  and 
Quadratic  Hopfield  Associative  Memories  (HAMS) . 

Extensive  computer  simulations  were  conducted  to  explore  the 
characteristics  of  both  first  and  higher  order  Hopfield  asso¬ 
ciative  memories.1  Based  on  the  simulation  results,  a  new  statis¬ 
tical  method,  which  we  will  call  the  C  parameter  approach,  was 
developed  and  successfully  applied  to  derive  a  series  of  equations 
capable  of  precisely  estimating  the  memory  capacity  and  the  at¬ 
traction  radius  of  the  HAMS.2'3  The  origins  of  this  average  SNR 
parameter  come  from  the  signal-to-noise  ratio  concept  commonly 
seen  in  binary  decision  theory.  We  found  that  these  two  important 
performance  characteristics  of  HAMS  are  closely  related  to  the 
required  convergence  probability.  More  specifically,  the  required 
convergence  probability  of  the  HAM  must  be  explicitly  specified 
before  the  memory  capacity  can  be  quantified.  We  also  showed, 
through  our  single  parameter  approach,  that  the  memory  capacity 
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can  be  doubled  by  appropriately  weakening  the  required  convergence 
probability. 

There  are  two  significant  advantages  of  the  C  parameter 
approach:  (1)  it  is  much  simpler  than  any  statistical  approaches 
proposed  by  other  researchers4/5/6/7;  and  (2)  it  is  not  only 
capable  of  characterizing  the  performance  of  the  direct  conver¬ 
gence  HAM  in  which  the  network  is  required  to  converge  precisely 
to  the  target  vector  in  one  iteration,  but  it  is  also  capable  of 
characterizing  the  indirect  convergence  HAM  in  which  a  small 
fraction  e  of  its  N  components  are  permitted  to  have  errors  even 
after  multiple  iterations  (whether  performed  synchronously  or 
asynchronously) .  An  important  result  we  found  associated  with  the 
indirect  convergence  HAM  is  that  there  exists  a  one-to-one  corre¬ 
spondence  between  P^c  (the  required  indirect  convergence  probabil¬ 
ity)  and  e/rj ,  where  v  is  the  probability  that  a  particular  neuron 
changes  to  an  incorrect  bit  after  one  updating  cycle.  For  exam¬ 
ple,  when  €=0.06,  v=0. 000755  and  e/ri=79 ,  we  have  P^c=0.994.  If 
one  tightens  the  allowance  of  input  noise  (i.e.  smaller  e)  to  keep 
Pic  at  0.994,  either  the  number  of  stored  patterns  allowed  (i.e. 

M)  or  the  maximum  noise  immunity  (i.e.  attraction  radius)  has  to 
be  decreased.  Doing  so  is  equivalent  to  reducing  the  value  of  V 
to  the  extent  that  we  keep  e/r?=79.2/3  In  fact,  there  also  exists 
a  similar  relationship  for  the  direct  convergence  HAM  in  which  a 
constant  value  of  N  corresponds  to  a  constant  P^c  (the  required 
direct  convergence  probability) . 
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The  concept  of  the  memory  capacity  of  the  HAM  can  be  best 
explained  by  the  equation:  M1  =  (1  -  2p)P  M,3  where  p  =  2b/n,  b  is 
the  number  of  noisy  input  bits;  m'  is  the  number  of  stored 
patterns  for  the  noisy  case  b^O,  while  M  is  the  capacity  for  the 
case  b=0.  Finally  p  is  the  order  of  the  HAM,  i.e.  p=l  is  the 
first  order  HAM,  p=2  is  the  quadratic  HAM,  etc.  Therefore,  one 
can  think  of  the  recall  power  of  an  arbitrary  HAM  being  jointly 
characterized  by  the  number  of  stored  patterns  M  and  the  desired 
noise  immunity  of  the  HAM. 

The  power  of  the  C  parameter  was  further  illustrated  by  its 
capability  of  providing  satisfactory  theoretical  explanations  for 
the  performance  differences  between  the  (-1,1)  and  (0,1)  binary¬ 
valued  HAMS  and  similarly  between  the  nonzero-autoconnection  and 
zero-autoconnection  HAMS.  The  trade-offs  associated  with  the 
inclusion  of  the  binary  correlation  matrix  in  the  quadratic  HAM 
was  also  investigated  using  this  same  technique. 

2.  Electro-Optical  Implementation  of  a  Quadratic  Neural  Net¬ 
work. 


During  the  first  year  of  funding,  our  approach  was  to  build 
upon  our  previous  designs  with  optical  polynomial  processors.8 
Using  the  approach  outlined  in  the  grant  proposal,  we  have  de¬ 
signed,  built  and  tested  a  processor  using  polarization  encoding. 
The  processor  implements  a  polynomial,  which  represents  the 
discriminant  function  of  perceptron-based  neurons,  as  the  outer 
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product  of  a  vector  x  and  followed  by  a  generalized  inner  product 
with  a  matrix  of  weights  (coefficients  of  the  polynomial)  W.  The 
result  is  obtained  as  the  element-by-element  multiplication  be¬ 
tween  the  outer  product  matrix  and  the  weight  matrix  followed  by 
spatial  integration.  The  architecture  is  shown  below. 


Si  PHOTODIODES 


Here  the  LCTV  modulators  1  and  2  perform  the  outer  product,  while 
LCTVs  3  and  4  perform  the  weighting  on  the  outer  product  matrix. 

By  using  the  properties  of  the  polarization-encoding  technique,  we 
have  been  able  to  reduce  the  space-bandwidth  product  to  one-fourth 
of  what  would  normally  be  required  in  systems  for  operating  with 
bipolar  numbers. 
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To  date  we  have  built  and  tested  a  system  that  implements  a 
single  neuron  (only  one  polynomial  can  be  evaluated  at  a  time).9 
We  are  working  to  implement  a  multineuron  system  by  using  spatial 
multiplexing  and  utilizing  a  two-dimensional  detector  (i.e.  a  CCD 
camera)  at  the  output.  In  addition,  we  have  applied  the  processor 
to  implement  optical  logic  and  perform  binary  and  trinary  valued 
transforms  such  as  the  Walsh  and  Haar.10  The  amount  of  memory 
required  for  storing  the  transform  matrix  has  been  drastically 
reduced  by  formulating  the  transform  as  an  outer  product. 

We  have  simulated  the  improvement  in  performance  of  the 
quadratic  neural  network  over  the  linear  neural  network  in  per¬ 
forming  the  EXOR  operation.  We  are  currently  simulating  the  per¬ 
formance  of  the  quadratic  network  in  pattern  recognition  tasks,  in 
particular  the  problem  of  character  recognition.  Depending  on  the 
results  of  these  simulations,  we  will  be  investigating  methods  for 
reducing  redundant  terms  in  the  outer  product  matrix. 

3.  Optical  Implementation  Using  Photorefractive  Crystals. 

As  part  of  our  research  on  optical  quadratic  neural  networks, 
we  have  developed  an  optical  quadratic  polynomial  processor  using 
four-wave  mixing  in  photorefractive  crystals.  This  processor  can 
perform  an  all  optical  quadratic  operation  for  use  as  a  decision 
function  for  quadratic  neural  networks  or  as  an  optical  polynomial 
processor.  Given  an  N-dimensional  input  vector  x  and  an  N  by  N 
weight  matrix  W,  the  processor  performs  the  quadratic  operation 
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xTWx  in  parallel.11  It  extends  earlier  work  on  matrix-vector 
multiplication  in  photorefractive  media  by  Yeh  and  Chiou.12  There 
are  a  number  of  advantages  of  a  photorefractive  quadratic 
processor.  First,  the  processor  performs  an  all  optical  parallel 
multiplication  and  summing  operation.  Second,  the  multiplication 
in  the  processor  is  a  coherent  multiplication.  Therefore,  the 
system  can  perform  quadratic  operations  on  complex  numbers  and 
phase-encoded  numbers.  Third,  with  very  low  powers  (10  uW  on  the 
input  elements) ,  the  processor  achieved  a  signal-to-noise  power 
ratio  of  20  at  the  output.  With  higher  input  powers,  the  signal- 
to-noise  ratio  will  increase  due  to  reduced  ambient  noise  effects. 
Fourth,  the  efficiency  of  photorefractive  processors  is  very  high. 
Due  to  gain  effects  associated  with  four-wave  mixing,  the  power  in 
the  output  beam  can  be  significantly  higher  than  that  in  the 
signal  beams.  This  helps  eliminate  a  common  fan-out  problem 
encountered  in  many  optical  processors. 

The  processor  has  been  implemented  using  a  4 -watt  argon  ion 
laser  as  the  light  source,  transparencies  as  the  vector  and  matrix 
input  modulators,  and  a  BaTi03  crystal  as  the  photorefractive 
processor.  Problems  associated  with  the  reduced  levels  of  power 
(approximately  300mW)  currently  available  from  our  4-watt  argon 
laser  will  soon,  we  believe,  be  overcome  with  the  planned  purchase 


of  a  new  laser. 
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